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JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2 **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l] The image store method which consists of the step which projects two or more 
images which photoed bearing which changed with digital cameras etc. on the spherical 
surface centering on lens cores, such as this digital camera, and sticks them from the same 
location, a step which carry out the **** projection of the image on this spherical surface at 
the flat surface which intersects this spherical surface, and a step which bundle up the 
data of the image by which **** projection was carried out at this flat surface, and store in 
memory or a record medium. 

[Claim 2] The step which projects two or more images which photoed bearing which 
changed with digital cameras etc. on the spherical surface centering on lens cores, such as 
this digital camera, and sticks them from the same location, the 2nd semi-sphere side by 
the side of the image on the 1st [ by the side of one station of this spherical surface ] 
semi-sphere side, and the pole of another side - this - the image of some [ which followed 
the 1st semi-sphere side ] tops The image store method characterized by consisting of a 
step which bundles up the data of the image by which carried out **** projection on the 
same flat surface including the equatorial plane of this spherical surface, and **** 
projection was carried out at this flat surface, and is stored in memory or a record medium. 
[Claim 3] The step which projects two or more images which photoed bearing which 
changed with digital cameras etc. on the spherical surface centering on lens cores, such as 
this digital camera, and sticks them from the same location, The step which carries out 
**** projection of the image on the 1st [ by the side of one pole of this spherical surface ] 
semi-sphere side at the 1st flat surface including the equatorial plane of this spherical 
surface, and carries out **** projection of the image on the 2nd [ by the side of the pole of 
another side of this spherical surface ] semi-sphere side at the 2nd flat surface including 
this equatorial plane, The image store method characterized by consisting of a step which 
bundles up the data of the image by which bundled up the data of the image by which **** 
projection was carried out at this 1st flat surface, and memorized in memory, and **** 
projection was carried out at this 2nd flat surface, and is stored in memory or a record 
medium. 

[Claim 4] The medium which is characterized by recording the program for making a 
computer perform each step of an image store method according to claim 1, 2, or 3 and 
which can be machine read. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to processing of the image expressed 
electronically, and relates to fields, such as lamination composition of the image which 
photoed bearing which is different especially with a digital camera etc. 
[0002] 

[Description of the Prior Art] Motion picture camera machines, such as a digital camera, 
have the effective approach of sticking the image of two or more sheets which photoed 
different bearing, and compounding it, in order to obtain the image of high resolution, the 
image of a wide angle, a panorama image, etc. more, since a visual field and resolution are 
limited. 

[0003] If a panorama image is taken for an example, now, two or more images will be stuck 
and only a unidirectional (direction on which it sees from a certain point and the circle on 
the celestial sphere is drawn) panorama image will be obtained. Drawing 6 explains an 
example of the creation technique of such a panorama image briefly. 

[0004] Each pixels (a, b, etc.) of the image G photoed with the digital camera from a certain 
point are projected on the cylinder side 300 which made the lens core c of a digital camera 
the medial axis, a' and b' are the points projected on the cy Under side 300 of Pixels a and b. 
The pixel of the image which carried out division photography of the 360'degree scene with 
the swing, and was photoed in the digital camera in the direction at least in all directions 
is projected [ that is, ] on the cylinder side 300, making parallel move the optical axis of a 
digital camera to the flat surface (usually level surface) containing Oa\ with the lens core c 
of a digital camera fixed. Thus, by projecting the image of two or more sheets on the 
cylinder side 300, and sticking it, a 360-degree panorama image is compoundable. In 
displaying this panorama image on the screen of a display, a flat surface vertical to the 
direction of a look to observe is defined, and it re-projects the pixel on the cylinder side 300 
on this flat surface. About an image data storage and management, by preparing the room 
corresponding to the rectangle or square which developed the cylinder side 300 where two 
or more images were projected, and making the information on all the pixels projected here 
on the cylinder side 300 memorize, the data of a 360-degree panorama image can be put in 
block, and it can memorize in memory, and can manage efficiently. 

[0005] About a unidirectional panorama image, it can compound by technique which was 
described above, and the image data can be put in block, and it can memorize and manage 
efficiently. However, the practical technique for compounding a three dimension panorama 
image which sticks two or more images which photoed bearing of the surrounding 
arbitration of a camera location, for example, covers the whole sky, and the image data 
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being put in block, and memorizing and managing efficiently is not well-known until now. 
[0006] 

[Problem(s) to be Solved by the Invention] The artificer etc. has already proposed the 
technique of projecting two or more images photoed with the digital camera etc. from the 
same point on the spherical surface centering on the lens core of a camera, and sticking 
them about composition of the three dimension panorama image described above 
(Japanese Patent Application No. No. 166737 [ nine to ]). 

[0007] The problem is left behind to preservation and management of image data when 
enforcing this technique. Namely, since it cannot develop simple like a cylinder side, by 
easy approach like the image data projected on the cylinder side, the spherical surface 
cannot bundle up the data of the image which projects on the spherical surface and was 
stuck, arid cannot save and manage them efficiently. 

[0008] The data of each image before composition are independently memorized in memory, 
and it is also possible to adopt the approach of projecting the image contained in the visual 
field of the bearing on the spherical surface, and re-projecting lamination and it on a 
display flat surface to observe an image about a certain bearing. However, by performing 
image composition, it is difficult at every display to take time amount by display, to change 
bearing continuously in a short time, and to observe an image. 
[0009] In order to make high-speed image display possible, image composition is 
beforehand performed about much bearing of fixed space include-angle spacing, the data of 
each synthetic image are independently memorized in memory, and how to display the 
synthetic image of bearing to observe is also considered. However, management of image 
data is troublesome to the top where the amount of memory required of this approach for 
an image data storage is large. Furthermore, the display of the image of bearing which is 
not prepared beforehand is difficult. 

[0010] Therefore, the object of this invention is to offer the approach of enabling efficient 
preservation and management, and the high-speed image rendering by the small amount 
of memory of the three-dimension-panorama image described above. 
[0011] 

[Means for Solving the Problem] This invention carries out **** projection of the image 
projected on the spherical surface to the flat surface which intersects the spherical surface, 
bundles up the data of the **** projection image, and stores them in memory or a record 
medium. 

[0012] According to this approach, the data of the three-dimension-panorama image which 
projects two or more images which photoed bearing of arbitration on the spherical surface, 
and stuck them from the same location with the digital camera etc., for example can be put 
in block, and it can save on memory or a record medium, and can manage as one file. If it 
does in this way, compared with the approach of saving the synthetic image of various 
bearings as a separate file, there will be few required amounts of memory far, arid it will 
end, and management of the image data on memory or a record medium will also become 
easy. Moreover, an image can be displayed on a high speed about bearing of arbitration by 
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re-projecting the image on memory or a record medium (already compounded panorama 
image) on the spherical surface, and projecting the required visual field part on a display 
flat surface further. 
[0013] 

[Embodiment of the Invention] Hereafter, one example of this invention is explained to a 
detail with reference to a drawing. 

[0014] Drawing 1 is a flow chart which shows the outline of processing by one example of 
this invention. Drawing 2 , drawing 3 , and drawing 4 are drawings for explanation of 
processing. 

[0015] First, in first step Si, it projects on the spherical surface 100 which set the lens core 
as Core O with the radius R equal to the distance to the lens and image pick-up side of a 
digital camera as shows two or more images (it is called an element image) which photoed 
bearing which is different from one point with a digital camera to drawing 2 , and 
lamination and one synthetic image are made on the spherical surface 100. In drawing 2 , 
Fl is one element image projected on the spherical surface 100, and p is the one pixel. N is 
the pole of the spherical surface which sees from Core O and exists right above, and calls it 
the north pole here. S is the pole of the spherical surface which sees from Core O and is just 
under, and calls it the south pole here. 102 is a vertical line which connects the north pole 
N and the south pole S, and calls it a polar axis here. 104 is a level field passing through 
Core O, and calls it an equatorial plane here. Q is the crossing of the longitude line and 
equatorial plane 104 which pass along the point that Pixel p was projected. Segment Ox is 
the datum line of LONG, for example, points out east bearing, phi is an include angle 
which Segment OQ makes to the datum line Ox, and calls it the LAT of Pixel p. theta is an 
include angle of the look and polar axis 102 which desire a pixel p from Core O to make. 
Generally, although the elevation angle from an equatorial plane is called LAT, the include 
angle theta with a polar axis 102 is called the LAT of Pixel p here, that is, theta= 90 
degree- it is [the LAT of general semantics]. 

[0016] At this step Si, the following coordinate operations are specifically performed. As 
for the location of each pixel of the element image photoed with the digital camera, it is 
common to be expressed with the rectangular coordinate system which made the optical 
axis (usually core) the zero. Then, the coordinate (i, j) of each pixel of an element image is 
once first changed into an azimuth coordinate (phix', phiyO by (l) type. However, in (l) type, 
R is the lens of the digital camera which photoed the element image and the distance of an 
image pick-up side, i.e., the radius of the spherical surface 100. Next, the azimuth 
coordinate of each pixel of an element image is changed into the azimuth coordinate (phi, 
theta) of the unified standard coordinates by (2) types. (2) In a formula, phi and theta are 
the LONG and the LAT which were explained in relation to drawing 2 . Moreover, A in (2) 
types is a transformation matrix between two system of coordinates. Such a coordinate 
transform matrix can be derived based on the relative location between the corresponding 
points of the overlap field of an adjacent element image as explained by the description of 
said Japanese Patent Application No. No. 166737 [ nine to ] in full detail. In order to ask 
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for coordinate-transform-matrix A, an element image is photoed so that the adjoining 
things may overlap selectively. 

[0017] In the following step S2, **** projection of the image projected and stuck on the 
spherical surface 100 is carried out at the flat surface 106 ( drawing 3 ) including an 
equatorial plane 104. Specifically, this **** projection is performed by changing the 
coordinate (phi, theta) of each pixel on the spherical surface 100 into a polar coordinate (r, 
phi) according to (3) types. 

[0018] Drawing 3 is the sectional view which cut the spherical surface 100 into round slices 
by the longitude line which passes along the projecting point and Point Q of Pixel p of 
drawing 1 . 106 is a **** projection flat surface including an equatorial plane 104. O' is the 
core of the element image Fl, and p' is a projecting point to the spherical surface 100 of the 
pixel p on the element image Fl. The point projected on the flat surface 106 at which this 
point p* includes an equatorial plane by **** projection is p." 

[0019] If aimed only at the scene above a horizontal line, all pixels will be projected on the 
interior of an equatorial plane 104 that what is necessary is just to carry out **** 
projection only of the image projected on the semi-sphere side above the equatorial plane 
104 of the spherical surface 100 (Northern Hemisphere side). However, the part below an 
equatorial plane 104 is also reflected to the element image F2 which took a photograph 
horizontally with the digital camera etc., and the pixel of such a part is projected by **** 
projection outside an equatorial plane 104 so that drawing 3 may see. For example, the 
pixel q of the element image F2 is projected on point q' on the spherical surface 100, and is 
further projected on q " of points on a flat surface 106 by **** projection, w is a pixel 
located in the bottom on the element image F2, is projected on point w 1 on the spherical 
surface 100, and is further projected on w M of points on a flat surface 106. A down scene 
also enables it only for a certain include angle to memorize the image data as one file not 
only from the scene on horizontal but from a horizontal direction in this example by 
extending the **** projection flat surface 106 from an equatorial plane 104. 
[0020] Drawing 4 is drawing for associating and explaining the image projection field and 
its image data storage field on the **** projection flat surface 106. **** projection of the 
pixel on the Northern Hemisphere side of the spherical surface 100 is carried out in the 
range shown with a circle 110. In order to project the pixel to the fixed range below the 
equatorial plane 104 on the spherical surface 100, a **** projection field is expanded to the 
field shown with a circle 112. Therefore, the field between a circle 110 and a circle 112 is a 
field where some (the Northern Hemisphere side is followed) images of the Southern 
Hemisphere side of the spherical surface 100 are projected. 

[0021] It returns to drawing 1 . In the last step S3, the data of the image in which **** 
projection was carried out by the before step S2 are collectively memorized in memory. The 
following actuation is performed and memory is made to specifically memorize image data 
by this example. 

[0022] First, the memory area corresponding to the square ABCD field on the **** 
projection flat surface shown in drawing 4 is secured. Although each pixel location by 
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which **** projection was carried out is expressed by the polar coordinate (r, phi) at the 
before step S2, generaUy a note is made and an integer address is attached to a location. 
Then, the coordinate of each pixel by which **** projection was carried out is changed into 
rectangular coordinates (X, Y) by (4) types. Next, a real number coordinate (X, Y) is 
changed into an integer coordinate (I, J) by (5) types. In addition, in (5) types, [x] means 
the maximum integer which does not exceed x. 

[0023] Thus, the pixel value or the brightness value f (I, J) calculated by the interpolation 
operation of for example, (6) types is written in the memory location of the address 
corresponding to the integer coordinate (I, J) searched for. (6) In a formula, f (Xi, Yj) is the 
pixel value or brightness value on the count in a real number coordinate (Xi, Yj). IpO is a 
interpolation function, for example, is a linear interpolation function and a cubic (Cubic) 
interpolation function. This interpolation operation is performed within the limits of the 
fixed distance around the observed integer coordinate (I, J) (for example, within the limits 
of (7) types) (however, Th constant). 

[0024] Thus, the data of the image which is projected on a part of Southern Hemisphere 
side which followed the Northern Hemisphere side of the spherical surface 100 and it, and 
was stuck can be put in block, and it can memorize in memory, and can manage as one file. 
[0025] Generally, in order to manage the data on memory efficiently by computer, it is 
desirable that it can express with a notation with the easy address range. In the case of a 
two-dimensional address (I, J), the management by A<I<B and the section like C<J<D is 
the most common. This example fills such a demand. 

[0026] Moreover, when treating image data, it is desired in respect of the homogeneity of 
data processing, or the precision of an image expression for the pixel which continues on 
memory to have spacing almost equal to spacing on the real space of the pixel on memory (I, 
J+l) and a pixel (I, J+2) on being spacing almost equal also on real space, i.e., the real 
space of the pixel on memory (I, J) and a pixel (I, I+l). Since the pixel arranged on memory 
is mostly distributed by the consistency of the same order with the pixel on the spherical 
surface (the difference of the consistency near a core and in the circumference is 1-4 at the 
maximum), it fills the upper demand with this example. 

[0027] What is necessary is to project the pixel on memory on the spherical surface 
according to the conversion process of reverse exactly with the case where an image is 
memorized, and just to re -project to a flat surface vertical to bearing to observe, in 
reproducing the image memorized by memory as mentioned above. Since composition of an 
element image has ended, it can reproduce the image of bearing of arbitration at a high 
speed. 

[0028] In addition, in some cases, I want to include in the object range to a downward scene 
considerably horizontally to photo the scene from a high crest top or the roof of a 
skyscraper, and obtain the panorama image. I want to include in the object range to the 
scene of right under to photo a scene from a balloon or a HEL and obtain the panorama 
image. Thus, when it is difficult how much for it to be horizontally made the object of 
processing to a lower part, and to determine uniformly, it is good to carry out **** 
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projection of the image projected on the upper half (Northern Hemisphere side) of the 
spherical surface, and the image projected on the lower half (Southern Hemisphere side) 
independently in an equatorial plane, respectively, to memorize each **** projection image 
in memory, respectively, and to manage as one file. Even if such, it is reproducible with the 
process described above. This is also one example of this invention. 

[0029] Processing of this invention explained above is realizable with a program on the 
computer of a configuration of having connected CPU200 as shown in drawing 5 R> 5, 
memory 201, a display unit 202, the user input equipments (a keyboard, mouse, etc.) 203, 
the hard disk drive unit 204, the floppy disk drive unit 205, and the PC card reader 206 
grade by the system bus 207. For example, by the PC card reader 206, the data of a series 
of images which were photoed from a certain point and recorded on PC card 208 with the 
digital camera are read into a computer, and are stored in a hard disk drive unit 204. The 
program 210 for processing of this invention explained in relation to drawing 1 is read from 
the floppy disk 209 set in the floppy disk drive unit 205, is stored in a hard disk drive unit 
204, when required, is loaded to memory 201 and performed. The data of the image set as 
the object of processing at the time of activation of this program 210 are read into memory 
201 from a hard disk drive unit 204. When the memory area 211 required for processing, 
for example, the memory area where **** projection image data is written in at step S3 of 
drawing 1 , is required, it is secured on memory 201. The **** projection image data 
written in this memory area 211 is saved in a hard disk drive unit 204 or a floppy disk 209 
if needed, and is again read into memory 201 at the time of an image rendering. 
[0030] As stated above, according to this invention, the cutback of the amount of memory 
required for preservation of a three dimension panorama image etc. and a high-speed 
image rendering are attained, but the mutual comparison of a series of images further 
photoed from the different point and an operation become easy. For example, when 
comparing a series of images which photoed the surrounding scene at the point with a 
photography person with a series of images which moved the location and photoed the 
surrounding scene and measuring the far and near information on a photographic subject, 
as for a series of images photoed from both points, being expressed by the unified system of 
coordinates is desirable. Although this invention carries out the **** projection expression 
of a series of images photoed from one point, since bearing (parallax) of the photographic 
subject with which it corresponds in the image photoed from the different point 
perpendicularly if the level surface is taken for the periphery of projection system of 
coordinates and a certain bearing is taken to a reference axis for the central point of 
projection system of coordinates can be expressed directly as a difference of a coordinate, 
for example, the extract of far and near information becomes easy. 
[0031] 

[Effect of the Invention] Effectiveness, like the extract of the far and near information on 
the photographic subject by the mutual comparison of a series of images which efficient 
preservation and management, and the high-speed image rendering of a 
three-dimension-panorama image etc. of data of which compounded two or more images 
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which photoed different bearing from the same location were attained according to this 
invention as explained above, and were photoed from the different point becomes easy can 
be acquired. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing l] It is the flow chart which shows the outline of processing by one example of 
this invention. 

[Drawing 21 It is drawing for explaining the globular projection of an element image. 
[Drawing 3] It is the sectional view which cut drawing 2 into round slices by a certain 
longitude line. 

[Drawing 4] It is drawing associating and showing the **** projection field and memory 
area of an image. 

[Drawing 5l It is the block diagram showing an example of the computer for processing this 
invention. 

[Drawing 6] It is the explanatory view of the panorama image creation by the projection to 

a cylinder side. 

[Description of Notations] 

100 Spherical Surface 

102 Polar Axis 

104 Equatorial Plane 

106 **** Projection Flat Surface 

Fl, F2 Element image 

O Core 

N North pole 

S South pole 

phiLAT 

theta LONG 

p, q, w Pixel on an element image 

p\ q\ w' Projecting point to a spherical-surface top 

p q w" Point projecting [.**** ] 

200 CPU 

201 Memory 

202 Display Unit 

203 User Input Equipment 

204 Hard Disk Drive Unit 

205 Floppy Disk Drive Unit 

206 PC Card Reader 

207 System Bus 
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208 PC Card 

209 Floppy Disk 

210 Program 

211 Memory Area 
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